Automatic image-capturing apparatus, automatic image-capturing control method, image display system, image display method, display control apparatus, and display control method

ABSTRACT

An automatic image-capturing apparatus including an image-capturing unit configured to capture an image of a subject so as to obtain captured image data; an automatic captured-image obtaining unit configured to automatically obtain the captured image data obtained by the image-capturing unit without being based on an operation of a user; an inclination detection unit configured to detect an inclination in at least one of a pan direction and a tilt direction; and a control unit configured to control the obtaining operation performed by the automatic captured-image obtaining unit on the basis of information on the inclination in the pan direction or in the tilt direction, the inclination being detected by the inclination detection unit.

RELATED APPLICATIONS

This application is a divisional of and claims the benefit under 35U.S.C. §120 of U.S. patent application Ser. No. 14/505,959, titled“AUTOMATIC IMAGE-CAPTURING APPARATUS, AUTOMATIC IMAGE-CAPTURING CONTROLMETHOD, IMAGE DISPLAY SYSTEM, IMAGE DISPLAY METHOD, DISPLAY CONTROLAPPARATUS, AND DISPLAY CONTROL METHOD,” filed on Oct. 3, 2014 which is adivisional of and claims the benefit under 35 U.S.C. §120 of U.S. patentapplication Ser. No. 13/863,943, filed on Apr. 16, 2013, which is adivisional of and claims benefit under 35 U.S.C. §120 of U.S. patentapplication Ser. No. 12/263,665, filed on Nov. 3, 2008, which claimsbenefit under 35 U.S.C. §119 of Japanese Patent Application JP2007-288335, filed in the Japanese Patent Office on Nov. 6, 2007, eachwhich is hereby incorporated by reference in its entirety.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an automatic image-capturing apparatusfor automatically capturing an image of a subject, in particular, anautomatic image-capturing apparatus to be worn by a user. Furthermore,the present invention relates to an image display system that isconfigured to include such an automatic image-capturing apparatus and aninformation processing apparatus and that automatically displayscaptured images and to an image display method for use therewith. Inaddition, the present invention relates to a display control apparatusfor performing display control of automatically captured images and to adisplay control method for use therewith.

2. Description of the Related Art

For example, as a life-log camera, a camera for recording scenery thatis viewed by a user in daily life as image data in such a manner that acamera worn by the user automatically performs periodic image capturinghas been proposed. By using such a life-log camera, it is possible tokeep his/her activity history and memories as image data.

More specifically, the life-log camera is worn by a user in such amanner that, for example, it is hung from the neck using a neck strap orit is integrally formed with an eyeglass unit. Then, in such a state,image capturing is performed, for example, at intervals of a fixed timeperiod, and captured images are stored.

Examples of the related art include Japanese Unexamined PatentApplication Publication No. 10-282536.

SUMMARY OF THE INVENTION

The object of automatically capturing and storing images using thelife-log camera described above is to allow the user to confirm his/heractivity history and to recall memories by reproducing and displayingstored images.

Here, in view of necessity for capturing an image of daily scenery atfixed intervals in the manner described above, it is assumed that a lotof captured images are to be stored in the life-log camera. As aconsequence, for reproducing stored images, a usage is conceived inwhich, unlike a case in which images captured by a normal camera device(a camera device for receiving captured images by a shutter operation(including self-timer image capturing) of a user or the like are to bereproduced and displayed, stored images are advanced and displayed at acomparatively high speed (in a riffled manner).

However, since a life-log camera is made to capture an image in a statein which it is worn by a user, there is a problem in that theinclination of the apparatus main unit varies greatly due to the userwalking or carrying out an action. As a consequence, there is a problemin that the inclination regarding the content of the captured image mayvary greatly.

In a case where a comparatively large variation in inclination occurs,in the case that stored images are advanced and reproduced in the mannerdescribed above, a phenomenon in which reproduced images swingunnaturally occurs. In particular, with regard to captured images by alife-log camera, comparatively high-speed sending and reproduction areperformed in the manner described above. As a result, when swinging of areproduced image as a consequence of image inclination occurs, theprobability of making a user having the images feel uncomfortable isincreased.

It is desirable to reduce the occurrence of swinging of a reproducedimage, which may occur during the reproduction (during the display) ofimage data captured by an automatic image-capturing apparatus thatautomatically captures an image of a subject.

In particular, it is desirable to reduce swinging of a reproduced image,which is caused by an inclination in the pan direction (for example, inthe horizontal axis direction of the image) or in the tilt direction(for example, in the vertical axis direction of the image), which occurswhen image capturing is performed.

According to an embodiment of the present invention, there is providedan automatic image-capturing apparatus configured as described below.

That is, the automatic image-capturing apparatus includesimage-capturing means for capturing an image of a subject so as toobtain captured image data, and automatic captured-image obtaining meansfor automatically obtaining the captured image data obtained by theimage-capturing means without being based on the operation of a user.

Furthermore, the automatic image-capturing apparatus includesinclination detection means for detecting an inclination toward at leastone of a pan direction and a tilt direction.

Furthermore, the automatic image-capturing apparatus includes controlmeans for controlling the obtaining operation by the automaticcaptured-image obtaining means on the basis of information on theinclination in the pan direction or in the tilt direction, which isdetected by the inclination detection means.

Furthermore, in an embodiment of the present invention, the automaticimage-capturing apparatus is configured as described below.

That is, the automatic image-capturing apparatus includesimage-capturing means for capturing an image of a subject so as toobtain captured image data, and automatic captured-image obtaining meansfor automatically obtaining the captured image data obtained by theimage-capturing means without being based on the operation of a user,and inclination detection means for detecting an inclination in at leastone of a pan direction and a tilt direction.

Also, the automatic image-capturing apparatus includes control means forperforming control so that the information on the inclination in the pandirection or in the tilt direction, which is detected by the inclinationdetection means, is associated with the captured image data obtained bythe automatic captured-image obtaining means.

According to another embodiment of the present invention, there isprovided an image display system configured to include an automaticimage-capturing apparatus for automatically capturing an image of asubject and an information processing apparatus configured to performdata communication with the automatic image-capturing apparatus.

That is, the automatic image-capturing apparatus may includeimage-capturing means for capturing an image of a subject so as toobtain captured image data; automatic captured-image obtaining means forautomatically obtaining the captured image data obtained by theimage-capturing means without being based on an operation of a user; andinclination detection means for detecting an inclination in at least oneof a pan direction and a tilt direction.

Furthermore, the automatic image-capturing apparatus may includeimage-capturing-apparatus-side control means for performing control sothat the information on the inclination in the pan direction or in thetilt direction, which is detected by the inclination detection means, isassociated with the captured image data obtained by the automaticcaptured-image obtaining means.

The information processing apparatus may include display means forperforming image display, and information-processing-apparatus-sidecontrol means.

The information-processing-apparatus-side control means may perform anobtaining process for obtaining the captured image data that is capturedon the automatic image-capturing apparatus side and with which theinformation on the inclination is associated, a determination processfor determining whether or not the inclination of the captured imagedata is within a predetermined range on the basis of the information onthe inclination associated with the captured image data obtained by theobtaining process, and a display control process for performing controlso that captured image data in which the inclination has been determinedto be within the predetermined range on the basis of the determinationresult by the determination process is displayed on the display means.

According to another embodiment of the present invention, there isprovided a display control apparatus configured as described below.

That is, the display control apparatus according to the embodiment ofthe present invention is a display control apparatus for performingdisplay control of automatically captured images with which informationon the inclination in the pan direction or in the tilt direction isassociated, and includes control means for performing the followingprocessing.

That is, the control means may perform a determination process fordetermining whether or not the inclination of the captured image data iswithin a predetermined range on the basis of the information on theinclination associated with the captured image data as the automaticallycaptured image; and a display control process for performing control sothat the captured image data in which the inclination has beendetermined to be within the predetermined range on the basis of thedetermination result by the determination process is displayed onnecessary display means.

As described above, the automatic image-capturing apparatus (automaticimage-capturing control method) according to the embodiment of thepresent invention controls the operation for obtaining captured imagedata on the basis of the result in which the inclination in the pandirection or the tilt direction is detected.

Alternatively, the automatic image-capturing apparatus causes theinformation on the detected inclination in the pan direction or towardthe tilt direction to be associated with the captured image dataobtained by automatic image capturing.

Furthermore, the image display system (image display method) and thedisplay control apparatus (display control method) according toembodiments of the present invention performs control so thatinformation on the inclination in the pan direction or in the tiltdirection is associated with captured image data that is obtained byautomatic image capturing and thereafter, during image reproduction,captured image data in which the inclination is within a predeterminedrange is displayed on the basis of the information on the associatedinclination.

According to the embodiments of the present invention, the obtaining ofcaptured image data that is automatically captured by an automaticimage-capturing apparatus for which use in a state of being installed ina user, such as, for example, a life-log camera, is conceived, iscontrolled on the basis of the information on the detected inclinationin the pan direction or in the tilt direction, or the display thereof iscontrolled on the basis of the information on the inclination in the pandirection or in the tilt direction, thereby making it possible toreproduce and display only the captured image data in which theinclination is within a predetermined range.

If only the captured image data in which the inclination is within apredetermined range can be reproduced and displayed in the mannerdescribed above, it is possible to reduce swinging of a reproducedimage, which occurs when automatically captured image data isreproduced. As a result, it is possible to prevent occurrence of asituation in which a user is given an uncomfortable feeling as in therelated art.

According to the automatic image-capturing apparatus (automaticimage-capturing control method) that causes information on the detectedinclination to be associated with captured image data that is obtainedby, in particular, automatic image capturing and the image displaysystem (image display method) according to the embodiments of thepresent invention, unlike the case of controlling the reception of anautomatically captured image, it is possible to obtain all capturedimage data obtained at an automatic image capturing timing. That is, forexample, whereas, during sending reproduction, display control based oninformation on an inclination associated to reduce swinging of areproduced image is performed, display of another captured image datacan be performed as necessary.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A and 1B are exterior views of an automatic image-capturingapparatus according to an embodiment of the present invention;

FIG. 2 is a block diagram showing the internal configuration of anautomatic image-capturing apparatus according to a first embodiment ofthe present invention;

FIG. 3 illustrates an operation as the embodiment;

FIG. 4 is a flowchart showing a processing operation for calculating anaverage value of inclinations as processing operation that should beperformed to implement operation as the first embodiment of the presentinvention;

FIG. 5 is a flowchart showing a processing operation for performingimage capturing control on the basis of the determination result ofinclinations in pan and tilt directions as processing operation thatshould be performed to implement operation as the first embodiment ofthe present invention;

FIG. 6 is a block diagram showing the internal configuration of anautomatic image-capturing apparatus according to a second embodiment ofthe present invention;

FIG. 7 is a block diagram showing the internal configuration of anautomatic image-capturing apparatus according to a third embodiment ofthe present invention;

FIG. 8 illustrates a method of estimating a traveling direction;

FIG. 9 is a flowchart showing a processing operation for estimating atraveling direction as processing operation that should be performed toimplement operation as the third embodiment of the present invention;

FIG. 10 is a flowchart showing a processing operation for performingimage capturing control on the basis of the determination result ofinclinations in pan and tilt directions as processing operation thatshould be performed to implement operation as the third embodiment ofthe present invention;

FIG. 11 illustrates the outline of operation according to a fourthembodiment of the present invention;

FIG. 12 is a block diagram showing the internal configuration of apersonal computer used in the fourth embodiment of the presentinvention;

FIG. 13 is a flowchart showing a processing operation that should beperformed in correspondence with the recording time in the automaticimage-capturing apparatus as processing operation for implementingoperation of the fourth embodiment of the present invention;

FIG. 14 is a flowchart showing a processing operation that should beperformed in correspondence with reproduction time as processingoperation for implementing operation of the fourth embodiment of thepresent invention;

FIG. 15 is a flowchart showing a processing operation that should beperformed in correspondence with recording time in the automaticimage-capturing apparatus as processing operation for implementingoperation of a fifth embodiment of the present invention;

FIG. 16 is a flowchart showing a processing operation (in a case wherethe first and second embodiments are applied) that should be performedin correspondence with reproduction time as processing operation forimplementing operation of the fifth embodiment of the present invention;

FIG. 17 is a flowchart showing a processing operation (in a case wherethe third embodiment is applied) that should be performed incorrespondence with reproduction time as processing operation forimplementing operation of the fifth embodiment of the present invention;and

FIG. 18 illustrates a modification of calculating an average valueaccording to the fifth embodiment of the present invention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Embodiments of the present invention will be described below. Thedescription will be given in the following order.

1. First Embodiment (obtaining control: example in which two-axis gyrosensor is used)

2. Second Embodiment (obtaining control: example in which directionsensor is used to detect pan direction)

3. Third Embodiment (obtaining control: example in which referenceinclination in pan direction is set as traveling direction)

4. Fourth Embodiment (display control: example in which average valueand traveling direction are calculated in real time)

5. Fifth Embodiment (display control: example in which average value andtraveling direction are calculated in an ex-post manner duringreproduction)

6. Modification

First Embodiment Example of Exterior of Automatic Image-CapturingApparatus

FIGS. 1A and 1B show examples of the exterior of an image-capturingapparatus 1 as an embodiment of the present invention.

FIG. 1A shows an example of the exterior of the image-capturingapparatus 1 of a neck-strap type. The image-capturing apparatus 1 inthis case has a part with which, for example, a strap is attached, andis mounted in such a manner that the strap is attached in that part andis hung around a user's neck as shown in the figure. The user may wearit so that an image-capturing lens 3L provided in the image-capturingapparatus 1 can perform image capturing by assuming toward the front ofthe user to be a subject direction.

Although not shown in the figure, in this case, for example, a displayunit (display unit 5 (to be described later)) used to monitor andreproduce captured images is provided in the back part of theimage-capturing apparatus 1.

FIG. 1B shows an example the exterior of the image-capturing apparatus 1that is formed as an eyeglass-type display camera. The image-capturingapparatus 1 in this case has, for example, a mounting unit of a framestructure that half-circles from both-side head part toward the backhead part, and is attached to the user as a result of being hung aroundboth auricles, as shown in the figure.

In the image-capturing apparatus 1 in this case, in a state in which theuser has worn the image-capturing apparatus 1, the image-capturing lens3L is arranged toward the front in such a manner that image capturing isperformed by assuming the field-of-view direction of the user to be asubject direction.

Furthermore, in the worn state shown in the figure, a pair of displayunits 5 and 5 for the left eye and the right eye are arrangedimmediately before both eyes of the user, that is, at a place where thelenses in normal eyeglasses are positioned. For the display units 5 inthis case, for example, a liquid-crystal panel is used, and bycontrolling transmittance, a through state shown in the figure, that is,a transparent or semi-transparent state can be formed. As a result ofthe display units 5 being made to be a through state, even if the userwears the image-capturing apparatus 1 like eyeglasses at all times, noproblem is posed in ordinary life.

A pair of display units 5 are provided so as to correspond to both eyesand also, one display unit 5 may be provided so as to correspond to oneeye. In addition, the display units 5 may not be provided.

In FIGS. 1A and 1B, the neck-hanging type or eyeglass-typeimage-capturing apparatus 1 has been shown. Various configurations forthe user to wear the image-capturing apparatus 1 are also possible. Anytype of mounting unit, such as, for example, a headphone-type,neck-band-type, behind-the-ear type, or hat type may be worn by theuser. In addition, a form may also be possible in which theimage-capturing apparatus 1 is worn by the user in such a manner thatthe image-capturing apparatus 1 is mounted in ordinary eyeglasses, avisor, a headphone, or the like by using fixture such as a clip.

Furthermore, the image-capturing apparatus 1 may not necessarily bemounted in the head part of the user.

In the case of FIG. 1A, the image-capturing direction is set as beingtoward the front of the user. Alternatively, the image-capturingapparatus 1 may also be worn in such a manner as to be hung around theneck so that image capturing is performed for the area behind the userwhen it is worn.

Then, in the case of FIG. 1B, the image-capturing direction is set as afield-of-view direction. In addition, a configuration in which theimage-capturing lens 3L is mounted so that image capturing of an areabehind the user, an area at a side of the user, an area above the user,an area in a downward direction toward the feet, and the like when theimage-capturing apparatus 1 is worn, and a configuration in which aplurality of image-capturing systems whose image-capturing directionsare the same or different are provided are possible.

Furthermore, in FIGS. 1A and 1B, an image-capturing direction varyingmechanism with which the subject direction can be varied manually orautomatically may be provided for one or a plurality of image-capturinglenses 3L.

As an image-capturing apparatus for performing moving or still imagecapturing, of course, forms other than those shown in FIGS. 1A and 1Bare possible. For example, a device, an example of which is a mobilephone, a PDA (Personal Digital Assistant), or a portable personalcomputer, and which is provided with a function of an image-capturingapparatus, can be conceived as the image-capturing apparatus 1 of thepresent embodiment.

Furthermore, in the above various forms, for example, a microphone forcollecting outside sound may be provided, so that an audio signal thatis recorded together with image data can be obtained. Furthermore, aspeaker unit and an earphone unit for performing audio output may alsobe formed.

Furthermore, a light-emitting unit for performing illumination in thesubject direction in the form of, for example, a light-emitting diode(LED), may be provided in the vicinity of the image-capturing lens 3L,or a flash light-emitting unit for performing still image capturing maybe provided.

Example of Interior Configuration of Automatic Image-Capturing Apparatus

FIG. 2 is a block diagram showing the internal configuration of theimage-capturing apparatus 1 as a first embodiment of the presentinvention.

As shown in the figure, the image-capturing apparatus 1 includes asystem controller 2, an image-capturing unit 3, an image-capturingcontroller 4, a display unit 5, a display controller 6, an operationinput unit 7, a two-axis gyro sensor 8, a storage unit 9, a memory unit10, a bus 11, and a communication unit 12.

The system controller 2 is formed by a microcomputer including, forexample, a central processing unit (CPU), a read-only memory (ROM), arandom access memory (RAM), a non-volatile memory unit, and an interfaceunit, and serves as a controller for controlling the entireimage-capturing apparatus 1. The system controller 2 performs variouskinds of computation processes and transmission and reception of acontrol signal with each unit via the bus 11 in accordance with aprogram stored in the internal ROM or the like, and causes each unit toperform a necessary operation.

The image-capturing unit 3 includes an image-capturing optical system 3a, an image-capturing element unit 3 b, and an image-capturing signalprocessor 3 c.

The image-capturing optical system 3 a in the image-capturing unit 3 isprovided with a lens system including the image-capturing lens 3L shownin FIG. 1, an aperture, a zoom lens, a focus lens, and the like; and adriving system for causing the lens system to perform a focus operationand a zoom operation.

In the image-capturing element unit 3 b in the image-capturing unit 3, asolid-state image-capturing element array for detecting image-capturinglight obtained by the image-capturing optical system 3 a and forgenerating an image-capturing signal by performing photoelectricconversion thereon is provided. The solid-state image-capturing elementarray is formed as, for example, a charge-coupled device (CCD) sensorarray or a complementary metal-oxide semiconductor (CMOS) sensor array.

The image-capturing signal processor 3 c in the image-capturing unit 3includes a sample-hold/automatic-gain-control (AGC) circuit forperforming gain adjustment and waveform shaping on a signal obtained bysolid-state image-capturing elements, and a video A/D converter, andobtains captured image data as digital data.

Furthermore, a white-balance process, a luminance process, a colorsignal process, and the like are performed on the captured image data.

Image capturing is performed by the image-capturing unit 3 having theimage-capturing optical system 3 a, the image-capturing element unit 3b, and the image-capturing signal processor 3 c, and captured image datais obtained.

The image data obtained by the image-capturing operation of theimage-capturing unit 3 is processed by the image-capturing controller 4.

Under the control of the system controller 2, the image-capturingcontroller 4 performs processing, such as an image compression processfor compressing captured image data at various compression ratios, animage size conversion process, and an image format conversion process,and also performs a process for transferring captured image data to eachunit (display controller 6, storage unit 9, memory unit 10, andcommunication unit 12, etc.) connected via a bus 11 according to theoperation status.

On the basis of the instruction of the system controller 2, theimage-capturing controller 4 performs on/off control of animage-capturing operation in the image-capturing unit 3, a shutterprocess, driving control of a zoom lens and a focus lens of theimage-capturing optical system 3 a, control of the sensitivity and theframe rate of the image-capturing element unit 3 b, parameter control ofeach process of the image-capturing signal processor 3 c, and setting ofexecution processes.

For the configuration for performing display to a user in theimage-capturing apparatus 1, a display unit 5 and a display controller 6are provided.

The display unit 5 is provided with a display panel unit, such as aliquid-crystal display, and a display driving unit for driving theliquid-crystal display unit. The display drive unit is formed of a pixeldriving circuit for performing image display unit. The pixel drivingcircuit applies, at a predetermined horizontal/vertical driving timing,a driving signal based on a video signal to each of pixels arranged in amatrix in the display panel unit, so that display is performed.

Under the control of the system controller 2, the display controller 6drives the pixel driving circuit in the display unit 5 so as to performpredetermined display. For example, real-time monitor display of imagescaptured by the image-capturing unit 3, display of a reproduced imageregarding captured image data recorded in the storage unit 9, and thelike, are performed.

In order to perform these displays, for example, luminance leveladjustment, color correction, contrast adjustment, sharpness (contourenhancement) adjustment, and the like can be performed. Furthermore,generation of an expanded image such that part of image data is expandedor generation of a reduced image, image effect processes, such as softfocus, mosaic, luminance inversion, highlight display (enhancementdisplay) of part of an image, changing of atmosphere of the color of thewhole, and the like can be performed.

The operation input unit 7 has operation elements such as, for example,keys, buttons, and dials. For example, operation elements used inpower-supply on/off operation and operation related to automatic imagecapturing are formed. Furthermore, when image capturing in accordancewith shutter operation of a user is to be enabled in addition toautomatic image capturing, operation elements used in, for example,shutter operation, zoom operation, exposure setting operation,self-timer image-capturing operation, and the like as user operationrelated to image capturing may be formed.

The operation input unit 7 supplies information obtained from suchoperation elements to the system controller 2, and the system controller2 performs necessary computation processes and control corresponding tothese items of information.

The storage unit 9 is used to store various kinds of data, includingcaptured image data.

This storage unit 9 may be formed of a solid-state memory such as aflash memory, and may also be formed of, for example, a hard disk drive(HDD).

The storage unit 8, rather than being formed as an incorporatedrecording medium, may also be formed as a recording and reproductiondrive compatible with a portable recording medium, for example, a memorycard in which a solid-state memory is contained, an optical disc such asa compact disc (CD) or a digital versatile disc (DVD), a magneto-opticaldisc, a hologram memory, or the like.

Of course, both a built-in type memory, such as a solid-state memory oran HDD, and a recording and reproduction drive for a portable recordingmedium may be installed.

Under the control of the system controller 2, the storage unit 9 recordsand reproduces the captured image data and other various kinds of data.

The memory unit 10 is provided to temporarily store various kinds ofdata, and is formed of a data rewritable memory device, such as, forexample, a RAM or a non-volatile memory. For example, the memory unit 10can be used as a temporary storage area (work area with which imagesignal processing is performed) for captured image data for whichvarious image signal processing are performed.

The communication unit 9 is provided as a unit for performing datacommunication with various kinds of external devices. For example,transmission and reception of data may be performed with a serverapparatus (not shown). In that case, for example, network communicationmay be performed via a short-distance wireless communication withrespect to a network access point by using a method, such as, forexample, a wireless LAN or Bluetooth or, for example, wirelesscommunication may be performed directly with a server apparatus having acompliant communication function.

Furthermore, the communication unit 12 may be connected to a device,such as a personal computer, by using an interface, such as, forexample, a universal serial bus (USB) method, so that transmission andreception of data is performed.

The communication unit 12 enables, for example, captured image data thatis captured and stored in the storage unit 9 to be transferred to apersonal computer or other external devices.

Here, as an example, the communication unit 12 is assumed to be capableof performing data communication with external devices by using a wiredconnection interface, such as, for example, a USB method. In the figure,an interface terminal TI/F is a terminal to which a communication cablefor connection with the external device side is connected whenperforming data communication through such wired connection.

The two-axis gyro sensor 8 is provided to detect the inclination of theimage-capturing apparatus 1 in the pan direction and in the tiltdirection.

In this case, the two-axis gyro sensor 8 is disposed inside theimage-capturing apparatus 1 in such a manner that one of the detectionaxis directions thereof matches the horizontal direction of theimage-capturing element (captured image) and the other detection axisdirection matches the vertical direction of the image-capturing element.As a result, the inclination (rotational axis speed) in the pandirection (horizontal direction: side-to-side direction) and theinclination in the tilt direction (vertical direction: up-and-downdirection) are detected.

Detection signals indicating the inclinations in the pan direction andin the tilt direction, which are obtained by the two-axis gyro sensor 8,are supplied to the system controller 2.

The configuration of the image-capturing apparatus 1 of the embodimentis not limited to the configuration described with reference to FIG. 2,and of course, additions and deletions of various components may be madeaccording to operation examples and functions performed in practice.

Control of Obtaining Automatically Captured Image in Accordance withInclination of Pan and Tilt

The image-capturing apparatus 1 shown in FIG. 2 performs, as anautomatic image capturing function, an operation for automaticallyobtaining captured image data based on a captured image obtained by theimage-capturing unit 3. More specifically, on the basis of an automaticimage capturing timing set, for example, every fixed time period, thesystem controller 2 gives instructions for the image-capturingcontroller 4 and performs control of obtaining captured image data. Atthis time, on the basis of the automatic image capturing timing, thesystem controller 2 performs control so that compressed image data basedon the captured image data obtained by the image-capturing unit 3 isgenerated by the image-capturing controller 4, and the compressed imagedata is recorded in the storage unit 9.

Here, as can be understood from the description thus far, theimage-capturing apparatus 1 of the present embodiment is assumed toautomatically capture an image of a subject in a state in which theimage-capturing apparatus 1 is worn by the user. It is assumed that theautomatic image-capturing apparatus formed to be such a wearable type asa life-log camera described earlier is used for the user to keep his/heractivity history and memories as image data.

However, when the automatic image-capturing apparatus is assumed to beused as such a life-log camera, as described earlier, there is a highprobability that large variations in the inclination occur in theautomatically captured-image data as a consequence of the user moving(that is, the person carrying the camera), and when the captured imageis advanced and reproduced (displayed), a problem arises in that thereproduced image swings unnaturally. In particular, regarding a capturedimage by a life-log camera, since comparatively high-speed sending andreproduction is assumed to be performed as described above, suchswinging of a reproduced image has a high probability of making a userfeel uncomfortable.

At this point, what should be noted is that an inclination that occursin image data in the pan direction and in the tilt direction is verydifficult to correct. That is, regarding an inclination (inclinationthat occurs with the center of the image being a rotational axis) thatoccurs in the plane direction of image data, it is possible to correctthe inclination by cutting out an image at an appropriate angle by, forexample, trimming. However, when an inclination occurs in the pandirection and in the tilt direction, the captured image is no longersuch that the subject is captured from the front, and therefore, it isvery difficult to perform appropriate correction. In particular, when acomparatively large inclination occurs, a subject to be displayedtemporarily moves outside the image frame, with the result that there isa high possibility that an image (that is, a failed image) in which thesubject being captured is not recognizable is obtained.

Accordingly, in the present embodiment, during reproduction, by notdisplaying captured image data for which a comparatively largeinclination has occurred in the pan direction and in the tilt direction,reduction in the swinging of the reproduced image and also prevention ofmixed-in of a failed image are achieved.

In the first embodiment, the operation of obtaining an automaticallycaptured image is controlled on the basis of the detection signal by thetwo-axis gyro sensor 8, thereby not reproducing or displaying an imagein which a comparatively large inclination has occurred in the pandirection and in the tilt direction in the manner described above.

FIG. 3 illustrates the operation of the first embodiment of the presentinvention.

In FIG. 3, sensor output in the figure indicates an output signal by thetwo-axis gyro sensor 8. Here, only a detection signal for one of the pandirection and the tilt direction is shown as an example. Furthermore,the automatic image capturing timing in the figure is a timing at afixed period, such as, for example, at intervals of 10 or 5 seconds,which is set in advance as an automatic image capturing period.

First, in this case, for obtaining captured image data, basically, anautomatic image capturing timing takes priority. The obtaining of acaptured image in this case is basically performed in a case where theresult of a determination of whether or not the inclination is within apredetermined range for each automatic image capturing timing in thefigure indicates that the inclination is within the predetermined range(<1> in the figure).

In this case, when it is determined that the inclination is not withinthe predetermined range at an automatic image capturing timing,monitoring of sensor output is started, and thereafter, the capturedimage data at the time the inclination becomes within the predeterminedrange is obtained (<2> in the figure).

Here, in the present embodiment, when it is to be determined whether ornot the inclination is within the predetermined range, the average valueA of sensor outputs is calculated.

That is, when the magnitude of the inclination is to be determined, itis necessary to define a direction (that is, a direction in which theuser is directed) serving as a reference. Here, the direction in whichthe front of the image-capturing apparatus 1 is directed in a state inwhich such average value A of the sensor outputs has been obtained isassumed to match the direction in which the user is directed, and themagnitude of the inclination is determined from the sensor outputs byusing the average value A as a reference.

More specifically, when the calculation of the average value A is to beperformed, the system controller 2 holds the values of sensor outputsobtained for the past several points in time. Then, on the basis of aplurality of sensor output values that are held in this manner and thesensor output value that is obtained at the current time, an averagevalue of the sensor outputs is calculated. That is, as a result, theaverage value A (the average value A of the inclinations) of sensoroutputs for the past predetermined period in which the current time isused as a reference is calculated.

Then, on the basis of the sensor output average value A that iscalculated in this manner and a preset threshold value a, it isdetermined whether or not the current inclination value is within apredetermined range defined using the value of the threshold value a.More specifically, it is determined whether or not the sensor outputvalue obtained from the two-axis gyro sensor 8 is within the range of ±ain which the average value A is used as a reference.

Here, in FIG. 3, for convenience of description, the captured imageobtaining control is described as being performed in accordance with thedetermination result based on the sensor output of one of the pandirection and the tilt direction. In practice, however, an average valueA is calculated with respect to sensor outputs of both the pan directionand the tilt direction in a similar manner. When it is determined thatthe inclination is within the predetermined range in both directions onthe basis of the result in which the same determination is performed onthe basis of the average value A and the threshold value a, control isperformed so that captured image data is obtained.

As a result of performing captured image obtaining control based on suchsensor outputs and the average value A thereof, in the image-capturingapparatus 1, as automatically captured images, only images in which theinclinations in the pan direction and in the tilt direction are withinthe predetermined range can be obtained. That is, as a result, duringreproduction, only images in which the inclinations in the pan directionand in the tilt direction are within the predetermined range can bereproduced and displayed, making it possible to reduce the swinging of areproduced image.

As a result of the swinging of the reproduced image being reduced, it ispossible to prevent the occurrence of a situation in which a user ismade to feel an uncomfortable as in the related art.

Processing Operation

A description will be given below, with reference to the flowcharts ofFIGS. 4 and 5, of processing operation that should be performed toimplement operation as the first embodiment described in the foregoing.FIG. 4 shows processing operation that should be performed to calculatean average value A of inclinations. FIG. 5 shows processing operationthat should be performed to perform control of obtaining a capturedimage on the basis of the average value A.

Processing operations shown in FIGS. 4 and 5 are performed by the systemcontroller 2 in accordance with, for example, a program stored in theinternal ROM. Furthermore, the system controller 2 is assumed to performprocessing operations shown in FIGS. 4 and 5 in parallel.

Processing operation shown in FIG. 4 will be described first.

In FIG. 4, in step S101, the pan and tilt inclination information isobtained. That is, the values of the detection signals for both the pandirection and the tilt direction, which are supplied from the two-axisgyro sensor 8, are obtained, and these values are stored in, forexample, an internal non-volatile memory or the like.

In the subsequent step S102, the pan and tilt inclination average valueis updated. That is, an average value of the values obtained and heldwithin predetermined period in which the current time is used as areference among the values of the detection signals obtained and held inthe past by the process of step S101, and the values of the detectionsignals obtained in step S101 at the current time is calculated withrespect to each of the pan direction and the tilt direction. Then, usingeach of the average values calculated in the manner described above, thevalue of the average value A (denoted as Ap) of the inclination in thepan direction and the value of the average value A (denoted as At) inthe tilt direction at the current time are each updated.

When the process of step S102 is performed, as shown in the figure,“RETURN” is reached.

Next, a description will be given of processing operation shown in FIG.5.

In FIG. 5, first, in step S201, a process for waiting for an automaticimage capturing timing is performed. That is, operation waits until theautomatic image capturing timing that is determined in advance, at whichthe captured image data obtained by the image-capturing unit 3 should beobtained, is reached.

Then, when the automatic image capturing timing is reached, in stepS202, a process for obtaining pan and tilt inclination information isperformed.

In the next step S203, a process for obtaining average values of pan andtilt inclinations is performed. That is, the information on the averagevalue Ap of the inclinations in the pan direction and the information onthe average value At of the inclinations in the tilt direction at thecurrent time, which are successively updated by the processing operationshown in FIG. 4 described above, are obtained.

In the subsequent step S204, it is determined whether or not theinclination is within the range with respect to both the pan and tiltdirections. That is, with respect to the pan direction, it is determinedwhether or not the value of the inclination in the pan directionobtained in step S202 is within the range of an average value Ap±a. withrespect to the tilt direction, it is determined whether or not the valueof the inclination in the tilt direction, which is obtained in stepS202, is within the range of the average value At±a.

When an affirmative result is obtained in step S204 by determining thatboth pan and tilt are within the range because the inclinations in thepan direction and the tilt directions are within the range in which theaverage values Ap and At are used as references, respectively, theprocess proceeds to step S205, where a process for controlling obtainingof a captured image is performed. Instructions are given to theimage-capturing controller 4, so that control is performed in such amanner that compressed image data of the captured image data obtained byimage-capturing unit 3 is generated and the compressed image data isrecorded in the storage unit 9.

When the obtaining control process in step S205 is performed, “RETURN”is reached as shown in the figure.

On the other hand, when, in step S204, a negative result is obtained bydetermining that at least one of inclinations of the pan direction andthe tilt directions is not within the range in which the average value Ais used as a reference, and both pan and tilt are not within the range,the process returns to step S202, as shown in the figure. That is, as aresult, hereinafter, obtaining of inclination information→obtaining ofan average value A→determination of whether or not the inclination iswithin the range are repeated until both pan and tilt are within therange. As a result, when the inclination is not within the predeterminedrange at the automatic image capturing timing, captured image data atthe time the inclination becomes within the predetermined rangethereafter is obtained.

Although a description using the drawings is omitted, examples of theprocessing operations shown in FIGS. 4 and 5 include operation input ofswitching off the power supply of the image-capturing apparatus 1 andoperation input for instructing the stopping of an automatic imagecapturing operation, which are completed in response to a generation ofa stop trigger that is set in advance to stop the automatic imagecapturing operation. More specifically, the system controller 2 performsprocessing operations shown in these figures, and the above-describedprocessing operation for waiting for a stop trigger to be generated asprocessing operation that should be performed in parallel, and completesthe processing operations shown in FIGS. 4 and 5 in response to thegeneration of the stop trigger.

This point is common to all the processing operations of the systemcontroller 2 described with reference to each of the followingembodiments.

Second Embodiment

Next, a second embodiment of the present invention will be described. Inthe second embodiment, for means for detecting an inclination in the pandirection, a direction sensor is used.

FIG. 6 shows the internal configuration of an image-capturing apparatus20 according to a second embodiment of the present invention.

The exterior of the image-capturing apparatus described in each of thefollowing embodiments, including the second embodiment, is identical tothat described with reference to FIG. 1 above, and accordingly, arepeated description is omitted. Regarding the internal components,those that have already been described are designated with the samereference numerals, and descriptions thereof are omitted.

As shown in FIG. 6, in the image-capturing apparatus 20 of the secondembodiment, the two-axis gyro sensor 8 provided in the image-capturingapparatus 1 shown in FIG. 2 is omitted. A direction sensor 21 detects aninclination in the pan direction, and a gyro sensor (tilt-direction gyrosensor) 22 detects an inclination in the tilt direction. In this case,the tilt-direction gyro sensor 22 is provided in the image-capturingapparatus 20 in such a manner that the detection axis direction thereofmatches the vertical direction of the image-capturing element.

The direction sensor 21 detects geomagnetism and thereby outputs adetection signal indicating the direction that the image-capturingapparatus 20 is directed.

In this case, the direction in which the image-capturing lens 3L isdirected is defined to be toward the front of the image-capturingapparatus 20. In response, the direction sensor 21 is provided in such amanner that the detection axis direction thereof matches the directionin which the image-capturing lens 3L is directed. As a result, when theimage-capturing lens 3L is directed north, a value indicating north isobtained as a detection signal. When the image-capturing lens 3L isdirected west, a value indicating west is obtained as a detectionsignal.

On the basis of such a detection signal of the direction sensor 21, itis possible to obtain information on the direction of an inclination inthe pan direction.

In the case of the second embodiment, the same determination as in thefirst embodiment is performed with regard to an inclination in the tiltdirection. That is, the system controller 2 in this case successivelycalculates and updates the average value At on the basis of thedetection signal from the tilt-direction gyro sensor 22. Also, inparallel with this, the system controller 2 obtains a detection signalvalue from the tilt-direction gyro sensor 22 and obtains the averagevalue At at each automatic image capturing timing, and determineswhether or not the detection signal value in the tilt direction iswithin the range of the average value At±a.

This also applies generally to the pan direction as in the case of thefirst embodiment except that the detection signal value of the gyrosensor is changed to the detection signal value of the direction sensor21.

That is, in this case, also, an average value of detection signal valuesfor the past predetermined period in which the current time is used as areference is calculated. At this time, for example, in a case where theimage-capturing apparatus 20 has been swung nearly from the north westdirection to the north east direction with north being the center in acertain period, as an average value calculated from the detection signalvalues of the direction sensor 21 in that period, an average valueindicating approximately north is obtained. Similarly, in a case wherethe image-capturing apparatus 20 has been swung nearly from the northeast direction to the south east direction with east being the center ina certain period, as an average value calculated from the detectionsignal values of the direction sensor 21 in that period, an averagevalue indicating approximately east is obtained.

That is, as a result, also, by calculating the average value based onthe detection signal values of the direction sensor 21, it is possibleto define the direction in which the user is directed similarly to thecase in which the average value Ap for the detection signal values ofthe gyro sensor are calculated. That is, it is possible to define theinclination (inclination=0) serving as a reference when the magnitude ofthe swinging of the image is to be determined.

Based on this point, hereinafter, also, the average value calculatedwith respect to the detection signal values of the direction sensor 21is denoted as an average value Ap.

Whereas the system controller 2 in this case performs a process forsuccessively calculating and updating the average value Ap of thedetection signal values of the direction sensor 21, the systemcontroller 2 obtains, at each automatic image capturing timing, thevalue of the detection signal of the direction sensor 21 and the averagevalue Ap, and determines whether or not the obtained value of thedetection signal is within a predetermined threshold value (set also asa in this case) in which the average value Ap is used as a reference.That is, it is determined whether or not the inclination is within therange of the average value Ap±a.

Then, also, in this case, on the basis of the result of thedetermination for the inclination in the pan direction and in the tiltdirection, which is performed as described above, it is determinedwhether or not the inclination is within the range in both pan directionand the tilt direction. When it is determined that the inclination iswithin the range in both the pan direction and the tilt direction, thecaptured image data at that time is obtained. On the other hand, when itis determined that the inclination is not within the range in at leastone of the pan direction and the tilt direction or the inclination isnot within the range in both the pan direction and the tilt direction,captured image data at the time the inclination becomes within the rangein both the pan direction and the tilt direction thereafter is obtained.

As can be understood from the foregoing description, in this case, theprocessing operation that should be performed by the system controller 2is the same as that described earlier in the first embodiment (FIGS. 4and 5) except that an inclination detection signal in the pan directionserves as a detection signal from the direction sensor 21 and aninclination detection signal in the tilt direction serves as a detectionsignal from the tilt-direction gyro sensor 22. Therefore, the repeateddescription of processing operation that should be performed by thesystem controller 2 in this case with reference to the drawings isomitted.

Third Embodiment

A third embodiment is formed such that an inclination serving as areference in the pan direction is not set to be an average value Ap ofdetection signal values described thus far, but set to be a travelingdirection of the user, which is estimated from the shift of pastposition information.

FIG. 7 shows the internal configuration of an image-capturing apparatus25 according to a third embodiment of the present invention.

The image-capturing apparatus 25 is formed such that a position detector26 is further added to the configuration of the image-capturingapparatus 20 in the above-described second embodiment.

The position detector 26 includes, for example, a global positioningsystem (GPS) GPS receiving unit, and detects current positioninformation on the basis of the result in which communication with a GPSsatellite (not shown) is performed.

The system controller 2 in this case estimates the traveling directionof the user on the basis of the position information detected by theposition detector 26.

An example of a traveling direction estimation technique is shown inFIG. 8.

In FIG. 8, the traveling direction of the user is estimated on the basisof the shift of the position from the past to the current time. Morespecifically, the direction of the movement from the position detectedat the previous time to the position (current position) detected at thecurrent time is assumed to be the traveling direction of the user.

In this case, the determination of whether or not the inclination in thepan direction is within a predetermined range is performed by using thetraveling direction of the user, which is estimated in the mannerdescribed above, as a reference. That is, the system controller 2 inthis case obtains, at each automatic image capturing timing, the valueof the detection signal supplied from the direction sensor 21, anddetermines whether or not the direction (that is, toward the front ofthe image-capturing apparatus 25) specified using the detection signalvalue is within a predetermined angle range in which the estimatedtraveling direction is used as a reference.

In this case, for the system controller 2, a threshold value b has beenset in advance as a value for defining the predetermined angle range,and the determination of whether or not the inclination is within thepredetermined angle range is performed on the basis of the fact thatwhen the value indicating the estimated traveling direction is denotedas B, the detection signal value obtained from the direction sensor 21is within the range of B±b.

A description will be given for confirmation. In this case, also,similarly to the case of the above-described second embodiment, theaverage value At of inclinations in the tilt direction is calculated andupdated, and a determination of whether or not the inclination in thetilt direction is within a predetermined range in which the averagevalue At is used as a reference is performed at the automatic imagecapturing timing.

Then, when it is determined that the inclination is within thepredetermined range in both the pan direction and the tilt direction onthe basis of the results of such determinations for the inclination inthe tilt direction and for the inclination in the pan direction based onthe traveling direction, the captured image data obtained at theautomatic image capturing timing is obtained. When it is determined thatboth inclinations are not within the range, the captured image data atthe time both inclinations become within the range thereafter isobtained.

Processing Operation

FIGS. 9 and 10 show processing operations that should be performed toimplement operations as the third embodiment described in the foregoing.

FIG. 9 shows processing operation that should be performed to estimate atraveling direction. FIG. 10 shows processing operation that should beperformed to perform captured image data obtaining control on the basisof detection signals in the pan and tilt directions.

The processing operations shown in these figures are performed by thesystem controller 2 shown in FIG. 6 in parallel in accordance with, forexample, a program stored in an internal ROM.

Furthermore, in this case, a process for successively updating theinformation on the average value At is performed with respect to thetilt direction. This processing is the same as that described above withreference to FIG. 4, and accordingly, a repeated description thereof isomitted herein.

First, in FIG. 9, in step S301, position information is obtained. Thatis, the current position information detected by the position detector26 is obtained, and this information is stored in, for example, aninternal non-volatile memory.

In the subsequent step S302, it is determined whether or not theposition information has been updated.

When a negative result is obtained in step S302 by determining that theposition information obtained at the current time is not changed fromthe position information stored at the previous time and the positioninformation has not been updated, the process returns to step S301 asshown in the figure, where position information for the next time isobtained. That is, since it is difficult to estimate the travelingdirection unless the position information is not updated, in that case,operation waits until the position information is updated.

On the other hand, when an affirmative result is obtained in step S302by determining that the position information obtained at the currenttime has been changed from the position information that was obtainedand stored at the previous time and the position information has beenupdated, the process proceeds to step S303, where a process fordetecting the direction of the movement from the previous position tothe current position is performed.

That is, the direction of the movement from the position indicated bythe position information obtained and stored in the process of step S301above, which was performed immediately before, to the position indicatedby the position information obtained in step S301 above is detected.

In the subsequent step S304, a process for updating traveling directioninformation is performed. That is, on the basis of the information onthe movement direction detected in step S305 above, the information onthe traveling direction of the user corresponding to the current time isupdated. When the process of step S304 is performed, “RETURN” isreached.

Next, processing operation shown in FIG. 10 will be described.

In FIG. 10, in step S401, similarly to step S201 above of FIG. 5,operation waits until an automatic image capturing timing is reached.Then, when the automatic image capturing timing is reached, in stepS402, a process for obtaining pan (azimuth) and tilt inclinationinformation is performed. That is, the value of the detection signalfrom the direction sensor 21 is obtained with respect to the pandirection, and the value of the detection signal by the tilt-directiongyro sensor 22 is obtained with respect to the tilt direction.

In the subsequent step S403, a process for obtaining traveling directioninformation and a tilt inclination average value is performed. That is,the information on the traveling direction that is successively updatedby the processing operation described above with reference to FIG. 9 isobtained and also, the information on the average value At ofinclinations in the tilt direction, which is successively updated byparallel processing (not shown), is obtained.

In the next step S404, a determination process is performed as towhether or not both pan and tilt are within the range.

That is, with respect to the pan direction, it is determined whether ornot the value of the detection signal from the direction sensor 21obtained in step S402 is within the range of B±b in which the value B ofthe traveling direction information obtained in step S403 is used as areference. In other words, it is determined whether or not the directionspecified using the value of the obtained detection signal from thedirection sensor 21 is within the range of a predetermined angle rangein which the direction indicated by the traveling direction informationis used as a reference.

Furthermore, with respect to the tilt direction, it is determinedwhether or not the value of the detection signal from the tilt-directiongyro sensor 22, which is obtained in step S402, is within the range ofAt±a in which the average value At obtained in step S403 is used as areference.

When an affirmative result is obtained in step S404 above by determiningthat both pan and tilt are within the range on the basis of the resultsof the determinations in the pan direction and in the tilt direction,the process proceeds to step S405, where a captured image obtainingcontrol process is performed.

When the obtaining control process of step S405 is performed, “RETURN”is reached as shown in the figure.

On the other hand, when a negative result is obtained in step S404 bydetermining that both pan and tilt are not within the range because atleast one of the values of the detection signals is not within the rangein the pan direction and in the tilt direction, the process returns tostep S402 above as shown in the figure. As a result, in this case, also,obtaining of information on direction and tilt inclination→obtaining oftraveling direction and average value At →a determination of whether ornot being within the range are repeated until both pan and tilt becomewithin the range. The captured image data at the time both theinclinations become within the predetermined range is obtained.

Fourth Embodiment

At this point, in each of the embodiments described thus far, swingingof a reproduced image with regard to an automatically captured image isreduced by not obtaining an obtained image in which an inclination isnot within a predetermined range. A fourth embodiment (and a fifthembodiment to be described later) is formed such that such control ofobtaining images is not performed, a determination of whether or not aninclination is within a predetermined range is performed duringreproduction, and image display control is performed on the basis of theresult, thereby reducing swinging of a reproduced image.

In particular, the fourth embodiment is formed such that an averagevalue A of inclinations is calculated at a timing at which a capturedimage is obtained by automatic image capturing or the travelingdirection is estimated.

Operation Outline

FIG. 11 illustrates operation outline of the fourth embodiment of thepresent invention.

Part (a) of FIG. 11 schematically shows operation (hereinafter referredto also as operation during recording) performed by the image-capturingapparatus in this case at an automatic image capturing timing.

Here, in the fourth embodiment, a description will be given in asummarized form with respect to all patterns for the following cases: acase in which, with respect to an inclination in the pan direction, asin the case of the first embodiment, it is determined whether or not adetection signal value from the gyro sensor is within the range of anaverage value Ap±a; a case in which, as in the second embodiment, it isdetermined whether or not, with respect to the pan direction, thedetection signal value from the direction sensor 21 is within the rangeof the average value Ap±a of the detection signal values from thedirection sensor 21, and with respect to the tilt direction, it isdetermined whether or not the detection signal value from thetilt-direction gyro sensor 22 is within the range of the average valueAt±a; and a case in which, as in the third embodiment, with respect tothe pan direction, it is determined whether or not the directionspecified using the detection signal of the direction sensor 21 iswithin the range of a predetermined angle range (within the range ofB±b) in which the traveling direction estimated using the positioninformation is used as a reference, and with respect to the tiltdirection, it is determined whether or not the detection signal valuefrom the tilt-direction gyro sensor 22 is within the range of theaverage value At±a.

Based on this point, in the following, for the sake of description, thevalue of the detection signal by the two-axis gyro sensor 8 describedwith reference to the first embodiment or the value of the detectionsignal by the direction sensor 21 and the tilt-direction gyro sensor 22described with reference to the third embodiment will be collectivelyreferred to as “pan and tilt direction inclination information”.

Furthermore, the average value A calculated from detected signal valuesof the gyro sensor (two-axis gyro sensor 8 or the tilt-direction gyrosensor 22), the average value A calculated from detected signal valuesof the direction sensor 21, or the traveling direction estimated fromthe position information will be collectively referred to as “referenceinclination information”.

Furthermore, as can be understood from the foregoing description, in thecase of the fourth embodiment, for the configuration of theimage-capturing apparatus, one of the configurations described withreference to FIGS. 2, 6, and 7 described above is adopted. For theimage-capturing apparatus of the fourth embodiment, it is only necessaryfor the system controller 2 shown in FIGS. 2, 6 and 7 to change aprogram that is internally stored so that processing operation forimplementing operation as the fourth embodiment (to be described below)is performed.

Operation outline during recording in the image-capturing apparatus inthis case shown in part (a) of FIG. 11, will be described below.

In the image-capturing apparatus in this case, an automatically capturedimage, and pan and tilt inclination information and referenceinclination information at that time are recorded in such a manner as tobe associated with each other. That is, in this case, all the imagesobtained at the automatic image capturing timing are obtained. Then, theaverage value A serving as the reference inclination information iscalculated, and the traveling direction serving as the referenceinclination information are estimated. Also, at each automatic imagecapturing timing, the pan and tilt inclination information and thereference inclination information are obtained, and the pan and tiltinclination information and the reference inclination information thatare obtained in this manner are associated with the captured image dataobtained by automatic image capturing.

In this case, the storage destination of the obtained pan and tiltinclination information and inclination reference information is set tothe storage unit 9 in the same manner as for the captured image data. Asa result, in the storage unit 9, at each automatic image capturingtiming, the captured image data, and the pan and tilt inclinationinformation and the inclination reference information are associatedwith each other and are sequentially stored.

In the fourth embodiment, on the basis of the pan and tilt inclinationinformation and the reference inclination information that areassociated with each item of captured image data in the manner describedabove, a determination of whether or not the inclination is within apredetermined range is performed, and on the basis of the result, imagedisplay control is performed.

Here, as a presumption, as the image-capturing apparatus of theembodiment, the stored image can be displayed on the display unit 5provided in the image-capturing apparatus and also, the stored image canbe transferred to an external device, such as a personal computer in themanner described above and displayed thereby. That is, based on thispoint, for image display control based on the pan and tilt inclinationinformation and the reference inclination information that areassociated with each other as described above, there can be twopatterns: a case in which the image-capturing apparatus performs imagedisplay control and a case in which the external device side performsimage display control.

Part (b) of FIG. 11 schematically shows, as pattern 1, operation forwhen the image-capturing apparatus performs display control.

Part (c) of FIG. 11 schematically shows, as pattern 2, operation forwhen image display control is performed by an external device (in thiscase, a personal computer) on the basis of the captured image datatransferred (obtained) from the image-capturing apparatus side, and thepan and tilt inclination information and the reference inclinationinformation that are each associated the captured image data.

The content of the display control that should be performed is the samefor both patterns 1 and 2. That is, on the basis of the pan and tiltinclination information and the reference inclination information thatare associated with each item of captured image data belonging to thereproduction target range, in response to, for example, reproductioninstructions (display instructions) for an automatically captured imageon the basis of user operation, each of the image-capturing apparatus inthe case of pattern 1 and the personal computer in the case of pattern 2determines, for each item of captured image data, whether or not theinclination is within a predetermined range in both the pan and tiltdirections, and selects captured image data to be reproduced anddisplayed.

Then, control for the display means is performed so that only thecaptured image data that is selected in this manner, in which theinclinations in the pan and tilt directions are within the predeterminedrange is advanced, reproduced, and displayed.

Internal Configuration of Personal Computer

FIG. 12 shows the internal configuration of a personal computer 30 forperforming image display control during reproduction in such a manner asto deal with the case of pattern 2.

In FIG. 12, a CPU 31 performs overall control and computation processingof the personal computer 30 in accordance with an initiated program. Forexample, the CPU 31 performs operation in response to operation inputfrom an input unit 35 (to be described later), storage of a data file inan HDD 38, generation/update of management information, and others.

The CPU 31 performs transmission and reception of control signals anddata to and from each unit via a bus 32 shown in the figure.

A memory unit 33 shows a ROM, a RAM, a flash memory, and the like usedfor processing by the CPU 31 in a comprehensive manner. In the ROM ofthe memory unit 33, operation programs, a program loader, and the likeof the CPU 31 are stored. Furthermore, in the flash memory, variouskinds of computation coefficients, parameters used in programs, and thelike are stored, and in the RAM, a data area and a task area with whicha program is executed are temporarily allocated.

In the HDD 38, under the control of the CPU 31, storage of data files,creation and update of management information, and the like areperformed.

In this case, in the HDD 38, an image obtaining and reproductionapplication 38 a by which the CPU 31 performs processing operation forimplementing obtaining of captured image data from the image-capturingapparatus, and pan and tilt inclination information and referenceinclination information associated therewith, and for implementingdisplay control performed on the basis of the pan and tilt inclinationinformation and reference inclination information is stored.

Furthermore, both the captured image data and pan and tilt inclinationinformation and reference inclination information obtained from theimage-capturing apparatus side can be recorded in the HDD 38.

The input unit 35 is formed as a keyboard, a mouse, a remote commander,and other input devices (not shown), and is provided to performoperation input. The information input via the input unit 35 issubjected to predetermined processing by an input processor 34 and istransferred as input of operation or data to the CPU 31. The CPU 31performs necessary computations and control in such a manner as tocorrespond to the input information.

A display unit 37 is formed to be, for example, a liquid-crystal displayunit, and displays various kinds of information to the user. Forexample, when the CPU 31 supplies display information to a displayprocessor 36 in response to various operation status, the input status,and the communication status, the display processor 36 drives thedisplay unit 37 on the basis of the supplied display data, therebyperforming display output for the various kinds of information.

Under the control of the CPU 31, a communication processor 39 performs atransmission data encoding process and a reception data decodingprocess.

An external communication interface 40 transmits transmission dataencoded by the communication processor 39 to an external device (in thiscase, the image-capturing apparatus) connected via a communication cableconnected to an interface terminal TI/F (for example, a USB terminalshown in the figure. Furthermore, the external communication interface40 transfers a signal transmitted from the external device, such as theimage-capturing apparatus, to the communication processor 39 via thecommunication cable.

The communication processor 39 transfers the received information to theCPU 31.

Obtaining Process

Here, as pattern 2 shown in part (c) of FIG. 11, in a case where thepersonal computer 30 side performs display control, of course, it isnecessary that captured image data stored in the image-capturingapparatus, and the pan and tilt inclination information and thereference inclination information associated therewith have beenreceived. For confirmation purposes, a description will be given of aprocess for implementing reception of information from theimage-capturing apparatus side.

First, on the personal computer 30 side, when the CPU 31 is connected tothe image-capturing apparatus side through an external communicationinterface, for example, a USB cable, the CPU 31 instructs the transferof each item of captured image data stored in the storage unit 9 of theimage-capturing apparatus, and the pan and tilt inclination informationand the reference inclination information that are associated with eachitem of captured image data.

On the image-capturing apparatus side, the system controller 2 performs,in response to the transfer instruction, reading control for the storageunit 9 and communication control for the communication unit 12 so thateach item of captured image data stored in the storage unit 9, and thepan and tilt inclination information and the reference inclinationinformation that are associated with each item of captured image dataare transferred to the personal computer 30 side.

In this case, the correspondence between each item of captured imagedata and the pan and tilt inclination information and the referenceinclination information can be understood also on the personal computer30 side as a result of, for example, management information indicatingthe correspondence being separately transferred.

On the personal computer 30 side, the CPU 31 performs control so thateach item of information transferred from the image-capturing apparatusside in the manner described above is recorded in the HDD 38. Thiscompletes obtaining of necessary information from the image-capturingapparatus side to the personal computer 30 side.

Regarding processing operation that should be performed so as to dealwith the reception time, which is described in the foregoing, processingoperation performed on the image-capturing apparatus side is performedby the system controller 2 in accordance with, for example, a programstored in the internal ROM, and processing operation on the personalcomputer 30 side is performed by the CPU 31 in accordance with theprogram as an image obtaining and reproduction application.

Furthermore, when transferring such information, in practice, after aprocess for confirming captured image data that has already beenobtained before information is transferred, instructions fortransferring information on only captured image data that has not beenobtained may be issued.

Processing Operation

Next, a description will be given, with reference to flowcharts in FIGS.13 and 14, of processing operations for implementing operations as thefourth embodiment described with reference to FIG. 11 above.

FIG. 13 shows processing operation that should be performed to implementoperation that is performed during recording by the image-capturingapparatus as the fourth embodiment described with reference to part (a)of FIG. 11 above. The processing operation shown in this figure isperformed by the system controller 2 in accordance with, for example, aprogram stored in an internal ROM.

In the image-capturing apparatus in this case, the processing operationfor successively updating reference inclination information is performedin parallel with the processing operation shown in this figure. For sucha process for updating reference inclination information, it is onlynecessary to perform one of the processes described with reference tothe first to third embodiments (FIGS. 4 and 9).

Referring to FIG. 13, in step S501, similarly to step S201 above,operation for waiting for an automatic image capturing timing isperformed. When the automatic image capturing timing is reached, in stepS502, a process for obtaining pan and tilt inclination information isperformed. That is, in the case of the configuration of theimage-capturing apparatus adapted to the first embodiment, a process forobtaining the values of detection signals in the pan direction and thetilt direction is performed by the two-axis gyro sensor 8. Furthermore,in the case of the configuration of the image-capturing apparatusadapted to the second or third embodiment, a process for obtaining thevalue of a detection signal from the direction sensor 21 and the valueof a detection signal from the tilt-direction gyro sensor 22 isperformed.

In the subsequent step S503, a process for obtaining referenceinclination information is performed. That is, in the case of theconfiguration of the image-capturing apparatus adapted to the firstembodiment, a process for obtaining an average value Ap and an averagevalue At based on detection signal values in the pan direction and inthe tilt direction by the two-axis gyro sensor 8, which are successivelyupdated by processing operation performed in parallel with theprocessing operation shown in this figure, is performed, respectively.Furthermore, in the case of the configuration of the image-capturingapparatus adapted to the second embodiment, a process for obtaining anaverage value Ap of detection signal values from the direction sensor 21and for obtaining an average value At of detection signal values fromthe tilt-direction gyro sensor 22, which are successively updated, isperformed.

Furthermore, in the case of the configuration of the image-capturingapparatus adapted to the third embodiment, a process for obtaininginformation on traveling direction estimated from the positioninformation that is successively updated by processing operationperformed in parallel with processing operation shown in this figure andfor obtaining an average value At of detection signal values from thetilt-direction gyro sensor 22 is performed.

In the next step S504, a captured image obtaining control process isperformed. That is, control is performed so that captured image data(compressed image data) is recorded in the storage unit 9 by a processidentical to that in step S205 of FIG. 5 above.

After that, in the next step S505, a process for associating obtainedimages with the obtained pan and tilt inclination information andreference inclination information is performed. For example, in thiscase, the correspondence between the captured image data; and the panand tilt inclination information and the reference inclinationinformation is managed using management information. In step S505,control is performed in such a manner that the pan and tilt inclinationinformation and the reference inclination information obtained in stepsS502 and S503 above are recorded in the storage unit 9. Also, a processfor updating information content of the management information in such amanner that the recorded pan and tilt inclination information andreference inclination information are associated with the captured imagedata recorded in step S504 is performed.

When the process of step S505 is completed, “RETURN” is reached as shownin the figure.

FIG. 14 shows processing operation that should be performed to implementoperation performed by the image-capturing apparatus during reproductionin the case of pattern 1 described with reference to part (b) of FIG. 11or to implement operation performed by the image-capturing apparatusduring reproduction in the case of pattern 2 described with reference topart (c) of FIG. 11.

As was also described above, image display control during reproductionapplies the same for pattern 1 and pattern 2. Based on this fact, withreference to FIG. 14, processing operation that should be performed bythe personal computer 30 will be described in a summarized manner.

In the case of pattern 1, the processing operation shown in this figureis performed by the system controller 2 in accordance with, for example,a program stored in the internal ROM. In the case of pattern 2, theprocessing operation shown in the figure is performed by the CPU 31 inaccordance with a program as the image obtaining and reproductionapplication 38 a shown in FIG. 12 above.

In FIG. 14, in the initial step S601, operation for waiting forreproduction start instruction is performed.

That is, operation for waiting for reproduction start instruction forautomatically captured image data, which is issued by, for example, useroperation input or the like from the operation input unit 7 (in the caseof pattern 1) and the input unit 35 (in the case of pattern 2) isperformed.

When the reproduction start instruction has been issued, in step S602, aprocess for determining a reproduction target range is performed. Thatis, when the reproduction start instruction detected in step S601 aboveinstructs the starting of reproduction of images inside a necessaryfolder, image data in the folder is determined to be in a reproductiontarget range. When the reproduction start instruction instructs thestarting of reproduction of all the images recorded in the storage unit9 or in the HDD 38, all those images are determined to be in areproduction target range. Then, in response to the reproduction targetrange being determined in this manner, the value of the total number Nof image data in the reproduction target range is obtained.

In the subsequent step S603, a process for setting an imageidentification value n to 1 (n=1) is performed. As is clear from thefollowing description, the image identification value n is a value forthe system controller 2 or the CPU 31 to count in order to identify eachitem of captured image data in the reproduction target range.

In the next step S604, pan and tilt inclination information andreference inclination information that are associated with image data(n) are obtained. That is, the image data (n) recorded in the storageunit 9 (pattern 1) or in the HDD 38 (pattern 2), and the pan and tiltinclination information and the reference inclination information thatare associated therewith are obtained.

In the subsequent step S605, it is determined whether both pan and tiltare within the range. That is, on the basis of the pan and tiltinclination information and the reference inclination informationobtained in step S604 above, a determination of whether or not theinclination in the pan direction is within a predetermined range, and adetermination of whether or not the inclination in the tilt direction iswithin a predetermined range are performed. Based on these determinationresults, it is determined whether or not the inclination is within thepredetermined range in both the pan direction and the tilt direction.

When an affirmative result that both pan and tilt are within the rangeis obtained in step S605 on the basis of the determination in the pandirection and in the tilt direction, the process proceeds to step S606,where image data (n) is set as a reproduction target image. Thereafter,the process proceeds to step S607.

On the other hand, when a negative result is obtained in step S605 bydetermining that the inclination is not within the range in at least oneof the pan direction and the tilt direction and both pan nor tilt arenot within the range, the process directly proceeds to step S607.

In step S607, it is determined whether or not n=N. That is, it isdetermined whether or not a determination as to inclinations has beencompleted for all the image data in the reproduction target range. Whenit is determined in step S607 that a negative result is obtained bydetermining n≠N, the value of the identification value n is incremented(n=n+1) in step S608, and then, the process returns to step S604. As aresult, the processing is repeated until a determination as to theinclinations for all the image data in the reproduction target range iscompleted.

On the other hand, when an affirmative result is obtained in step S607by determining that n=N, the process proceeds to step S609, wherereproduction and display control for the reproduction target image isperformed. That is, regarding image data that is set as reproductiontarget image by the setting process of step S606 within the image datain the reproduction target range determined in step S602 above, control(in the case of pattern 1) for the storage unit 9 and the displaycontroller 6 or control for the HDD 38 and the display processor 36 isperformed in such a manner that the reproduced image data issequentially advanced and displayed on the display unit 5.

When the process of step S609 has been performed, the processingoperation shown in this figure is completed.

At this point, according to the fourth embodiment as described above, itis possible to obtain all automatically captured images during recordingin view of reducing swinging of a reproduced image. That is, accordingto the fourth embodiment, whereas swinging of a reproduced image isreduced by display control, display of all the automatically capturedimages can be also performed as necessary.

For example, when obtaining control is performed as in the first tothird embodiments, even with an image that is deemed important, such asan image capturing a decisive moment, when the inclination is not withina predetermined range, the obtaining (recording) thereof is notperformed, with the result that it becomes not possible for the user tobrowse the image. However, according to the method of the fourthembodiment, since all automatically captured images can be recorded, theimage can be reproduced and displayed as necessary.

Fifth Embodiment

In the fourth embodiment described in the foregoing, the calculation ofthe average value A of inclinations and the estimation of a travelingdirection are performed in advance during recording in whichautomatically captured images are recorded. However, in a fifthembodiment of the present invention, the calculation of the averagevalue A and the estimation of the traveling direction are performedduring reproduction in an ex-post manner.

In the fifth embodiment, the configuration of the image-capturingapparatus and the configuration of the personal computer 30 is onlynecessary to be identical to those in the case of the fourth embodiment,and a repeated description thereof is omitted.

Flowcharts of FIGS. 15 to 17 show processing operations for implementingoperations as the fifth embodiment in which the calculation of theaverage value A and the estimation of the traveling direction areperformed in the manner described above during reproduction.

First, with reference to FIG. 15, a description will be given ofprocessing operation that should be performed by the image-capturingapparatus in this case during recording. The processing operation shownin FIG. 15 is performed by the system controller 2 in accordance with,for example, a program stored in an internal ROM.

In FIG. 15, in the initial step S701, operation for waiting for anautomatic image capturing timing is performed. When the automatic imagecapturing timing is reached, in step S702, a process for obtainingposition information and/or pan and tilt inclination information isperformed.

That is, when, as in the case of the first embodiment, the averagevalues Ap and At of inclinations in the pan and tilt directions are tobe calculated on the basis of the value of the detection signal of thetwo-axis gyro sensor 8, the values of the detection signals in the pandirection and in the tilt direction of the two-axis gyro sensor 8 areobtained. When, as in the case of the second embodiment, the inclinationaverage value Ap in the pan direction is to be calculated on the basisof the values of the detection signals by the direction sensor 21 andthe inclination average value At in the tilt direction is to becalculated on the basis of the values of the detection signals of thetilt-direction gyro sensor 22, the values of the detection signals fromthe direction sensor 21 and the tilt-direction gyro sensor 22 areobtained. Furthermore, when, as in the case of the third embodiment, thereference inclination of the pan direction is set to be a travelingdirection, values of detection signals from the direction sensor 21 andvalues of detection signals from the tilt-direction gyro sensor 22 areobtained. Furthermore, in this case, position information detected bythe position detector 26, which is necessary to estimate the travelingdirection, is also obtained.

In the subsequent step S703, captured image obtaining control isperformed. In the subsequent step S704, a process for associating anobtained image with the obtained position information and/or pan andtilt inclination information is performed. Here, the correspondencebetween the position information and/or the pan and tilt inclinationinformation; and the obtained image is managed using managementinformation. For this reason, in step S704, a process for updating thecontent of the management information in such a manner that the positioninformation and/or the pan and tilt inclination information obtained instep S702 above is recorded in the storage unit 9 and also the positioninformation and/or the pan and tilt inclination information isassociated with the captured image data recorded in step S703 above isperformed.

When the process of step S704 has been completed, “RETURN” is reached asshown in the figure.

Next, a flowchart in FIG. 16 shows processing operation that should beperformed on the image-capturing apparatus side or on the personalcomputer 30 side during reproduction.

Here, the processing during reproduction slightly differs between whenthe average value Ap is to be calculated as reference inclinationinformation in the pan direction and when the traveling direction of theuser is to be estimated. FIG. 16 shows processing operation that shouldbe performed in correspondence with a case in which the average value Apis to be calculated.

The processing operation shown in FIG. 16 is performed by the systemcontroller 2 in accordance with, for example, a program stored in aninternal ROM, or is performed by the CPU 31 in accordance with a programas the image obtaining and reproduction application 38 a (this appliesthe same for FIG. 17).

In FIG. 16, first, as a result of processing of step S801→step S803,similarly to the processing of step S601→step S603 of FIG. 14 above, areproduction target range and the value of the total number N of imageswithin the range are obtained in response to a reproduction startinstruction and also, the image identification value n is set to 1(n=1). In the next step S804, similarly to step S604 above, pan and tiltinclination information for image data (n) is obtained.

In this case, in step S805 subsequent to step S804 and in subsequentsteps, the calculation of the average value A, and a process for makinga determining as to the inclination on the basis of the average value Aare performed.

That is, first, in step S805, a process for obtaining pan and tiltinclination information of an image in a predetermined range in whichimage data (n) is used as a reference is performed. That is, pan andtilt inclination information associated with each item of image datathat is captured in a predetermined period that is more in the past thanthe automatic image capturing timing of the image data (n) within theimage data in the reproduction target range determined in step S802 isobtained.

In the subsequent step S806, average values of pan and tilt inclinationsare calculated. That is, by calculating those average values on thebasis of the pan and tilt inclination information obtained in step S804above and the past pan and tilt inclination information obtained in stepS805 above, the inclination average value Ap in the pan direction andthe inclination average value At in the tilt direction are computed.

In the next step S807, it is determined whether or not both pan and tiltare within the range.

When an affirmative result that both pan and tilt are within the rangeis obtained in step S807 on the basis of the results of thedetermination in the pan direction and the determination in the tiltdirection, the process proceeds to step S808, where a process forsetting image data (n) as a reproduction target image and then, theprocess proceeds to step S809.

On the other hand, when a negative result is obtained by determiningthat both pan and tilt are not within the range because at least one ofthe inclinations is not within the range, the process proceeds directlyto step S809.

In step S809, it is determined whether or not n=N. When a negativeresult is obtained by determining that n is not N, in step S810, thevalue of the identification value n is incremented (n=n+1) andthereafter, the process returns to step S804 above. As a result, theprocessing is repeated until the determination as to the inclinationbased on the average value is completed for all the image data in thereproduction target range.

On the other hand, when an affirmative result is obtained in step S809above by determining that n=N, the process proceeds to step S811, wherereproduction and display control for a reproduction target image isperformed similarly to step S609 above. When the processing of step S811has been performed, the processing operation shown in this figure iscompleted.

FIG. 17 shows processing operation that should be performed incorrespondence with a case in which the reference inclinationinformation of the pan direction is set as a traveling direction. Whenthe reference inclination information of the pan direction is set as atraveling direction, only the processing (steps S805 to S807) for aportion surrounded by a dashed line in FIG. 16 above differs. Theprocessing operation other than the portion surrounded by the dashedline is identical to those of FIG. 16. Accordingly, FIG. 17 shows onlythe processing of the extracted portions differing from those of FIG.16.

First, in this case, as processing in place of step S805 of FIG. 16,step S901 in the figure is performed. In step S901, a process forobtaining position information and tilt inclination information of animage in a predetermined range in which image data (n) is used as areference is performed. That is, with respect to the tilt direction,inclination information (the values of the detection signals of thetilt-direction gyro sensor 22) in the tilt direction, which is obtainedfor a predetermined period more in the past than the obtaining timing ofimage data (n), which is necessary to calculate the average value At, isobtained. With respect to the pan direction, position informationassociated with the image data (n), and position information associatedwith the image data (n) that was captured immediately before image data(n−1), which is necessary to estimate the traveling direction, areobtained.

In the subsequent step S902, a process for estimating the direction ofthe movement from the previous position to the current position isperformed. That is, on the basis of the position information of theimage data (n) obtained in step S901 above and the position informationof the image data (n−1), the direction of the movement from the previousposition to the current position is detected so as to estimate thetraveling direction of the user.

In the next step S903, the average value of tilt inclinations iscalculated. That is, on the basis of the inclination information in thetilt direction of the image data (n), which was obtained in step S804performed as a process preceding to step S901 above, and the inclinationinformation in the tilt direction for the past predetermined period,which was obtained in step S901 above, the average value At of theinclinations is calculated.

After that, in the next step S904, it is determined whether or not bothpan and tilt are within the range. That is, with respect to the pandirection, it is determined whether or not the direction of theinclination specified on the basis of the detection signal value fromthe direction sensor 21, which is obtained in step S804, is within apredetermined angle range (B±b) in which the traveling directionestimated in step S902 is used as a reference. Furthermore, with respectto the tilt direction, it is determined whether or not the value of thedetection signal from the tilt-direction gyro sensor 22, which isobtained in step S804, is within a predetermined range (At±a) in whichthe average value At calculated in step S903 is used as a reference.Then, on the basis of the results of these determinations, it isdetermined whether both values are within the range in both the pandirection and the tilt direction.

In a case where an affirmative result is obtained in step S904,processing operations performed in correspondence with the case in whicha negative result is obtained are identical to those in the case of FIG.16 above, and accordingly, a repeated description is omitted herein.

Here, according to the fifth embodiment, regarding processing thatshould be performed on the image-capturing apparatus side duringrecording, the average value calculation process and the travelingdirection estimation process can be omitted. In particular, as pattern2, when a display control process during reproduction is to be performedby an external device, such as the personal computer 30, on theimage-capturing apparatus side, it is only necessary to perform at leasta process for obtaining an automatically captured image, a process forobtaining position information and/or pan and tilt inclinationinformation, and a process for associating the obtained information withthe obtained image. That is, a processing burden on the image-capturingapparatus side can be correspondingly reduced more than in a case inwhich the method of each of the above-described embodiments is adopted.

If the processing burden is reduced, power consumption can also bereduced. In particular, an image-capturing apparatus in this case isassumed to be mainly used outdoors as a wearable-type automaticimage-capturing apparatus and is battery-driven. In this case, if powerconsumption is reduced, the driving time period of the apparatus can belengthened. As a consequence, image capturing for a longer period oftime is made possible, and the charging frequency of the battery can bereduced, making it possible to alleviate nuisance related to charging ofthe battery.

Modification

In the foregoing, the embodiments of the present invention have beendescribed. However, the present invention should not be limited to thespecific examples described thus far.

For example, in the description thus far, a case has been described inwhich an image in which both the inclination in the pan direction andthe inclination in the tilt direction are within a predetermined rangeis obtained or displayed has been exemplified. Alternatively, in a casewhere the inclination in one of the pan direction and the tilt directionis within a predetermined range, the image may be captured or displayed.

Furthermore, in the description thus far, as means for detecting aninclination in the tilt direction, only the gyro sensor is shown as anexample. Alternatively, for example, an inclination in the tiltdirection may be detected using another means, such as, for example, agravity sensor.

Furthermore, in the description thus far, a case in which, as adetermination of whether or not the inclination in the pan direction iswithin a predetermined range, only the determination based on adetection signal value by the gyro sensor is performed, or a case inwhich only the determination (two patterns of a case in which, asreference inclination information, the average value Ap of directions isused, and a traveling direction is used) based on detection signal valueby the direction sensor 21 is performed, has been exemplified.Alternatively, a plurality of determinations among them may be made, andwhen an affirmative result is obtained on the basis of at least aplurality of determinations, a final determination result that theinclination in the pan direction is within a predetermined range may beobtained.

By making a comprehensive determination based on detection signal valuesfrom different sensors in the manner described above, it is possible tomake an accurate determination in which an inclination that occurs inthe image-capturing apparatus in practice is reflected. This pointapplies the same for the tilt direction.

Furthermore, in the description thus far, it is presumed that the valuesof “a” and “b” that are set when a determination of whether or not theinclination is within a predetermined range is to be made are fixed.These values can be variable. For example, as a result of adetermination, when the number of images to be captured or displayed issmall, the values of “a” and “b” may be increased so that a largernumber of images are captured or displayed. Conversely, when the numberof images to be captured or displayed is large, the values of “a” and“b” may be decreased so that only the image with a smaller inclinationis captured or displayed.

Furthermore, in particular, in the fifth embodiment, when the averagevalue A is to be calculated in an ex-post manner during reproduction,during recording, only the pan and tilt inclination information obtainedat each automatic image capturing timing is recorded. Alternatively, notonly such pan and tilt inclination information obtained at eachautomatic image capturing timing, but also pan and tilt inclinationinformation obtained between automatic image capturing timings can berecorded.

By also recording pan and tilt inclination information obtained betweenautomatic image capturing timings in this manner, during reproduction,the average value A can be calculated by using many items of inclinationinformation obtained between automatic image capturing timings, and as aresult, the reliability of the average value A can be improved.

Here, specific processing content that should be performed incorrespondence with a case in which the average value A is to becalculated, including pan and tilt inclination information obtainedbetween automatic image-capturing timings in the manner described above,will be described. First, during recording, pan and tilt inclinationinformation obtained at each automatic image capturing timing may berecorded in association with obtained image in the same manner as in thedescription thus far. The pan and tilt inclination information obtainedbetween automatic image-capturing timings is separately recorded in sucha manner that the relationship between obtaining timings, including eachautomatic image capturing timing in the time axis, is shown.

Furthermore, during reproduction, when the average value A is to becalculated, an average value of the pan and tilt inclination informationfor a past predetermined period in which the obtaining timing (automaticimage capturing timing) of a target image is used as a reference,including the pan and tilt inclination information between automaticimage-capturing timings, which is separately recorded in the mannerdescribed above, may be calculated.

For the position information for estimating the traveling direction, theposition information obtained at a detection timing immediately beforeeach automatic image capturing timing in the position detector 26 isrecorded. As a result, the direction of the movement from the positionat the timing more immediately before the obtaining timing of the targetimage when compared to the case in which the traveling direction isestimated from the position information obtained at each automatic imagecapturing timing can be determined. As a result, it is possible to moreaccurately estimate the traveling direction.

Furthermore, in the fifth embodiment, when the average value A is to becalculated in an ex-post manner during reproduction, only the pan andtilt inclination information obtained in a period more past than theobtaining timing of the target image is used. Alternatively, the averagevalue A may be calculated including the pan and tilt inclinationinformation obtained at a timing in the past in the time axis than theobtaining timing of the target image.

At this point, an example thereof is shown in FIG. 18.

As shown in FIG. 18, the average value A may be calculated by using panand tilt inclination information associated with a predetermined numberof images after and before the target image (indicated using an obliqueline in the figure). More specifically, the average value A iscalculated using pan and tilt inclination information in a range of ±mimages in which the target image is used as a reference, which isindicated as a range “X” in the figure.

In this case, as shown as a shift of part (a) of FIG. 18→part (b) ofFIG. 18, each time an image for which the average value A is to becalculated is sequentially shifted, the center of the range X is shiftedfor every image (that is, every automatic image capturing timing).

In FIG. 18, when the average value A is to be calculated, a descriptionhas been given by using, as an example, a case in which only the pan andtilt inclination information (that is, only the pan and tilt inclinationinformation associated with each image) obtained at each automatic imagecapturing timing is used. Even in a case where pan and tilt inclinationinformation obtained between automatic image-capturing timings in themanner described above is used, the average value A may be calculated byusing pan and tilt inclination information obtained in a predeterminedperiod after and before an image for which the average value is to becalculated, which is used as a reference, in a similar manner.

In the manner described above, by calculating the average value A,including those in the past of the automatic image capturing timing ofthe target image, it is possible to obtain more accurate referenceinclination information.

Here, as can be understood from the description thus far in the fourthand fifth embodiments in which swinging of a reproduced image isprevented by display control during reproduction, during reproduction,it is determined whether or not the inclination is within apredetermined range at each automatic image capturing timing. In a casewhere such a method is to be adopted, in order that the image isdisplayed, it is necessary to satisfy the condition in which theinclinations in the pan and tilt directions were within a predeterminedrange at the automatic image capturing timing at which the image wascaptured. That is, this condition is a very stringent condition whencompared to the case in which image obtaining is controlled as in theabove-described first to third embodiments. As a result, in a case wherethe method described with reference to the above-described fourth andfifth embodiments is directly adopted, there is a risk that the numberof displayed images becomes extremely small.

Accordingly, as a countermeasure therefor, during recording, capturedimage data is obtained at a timing other than an automatic imagecapturing timing, and position information and/or pan and tiltinclination information at that time is obtained, making it possible todisplay a larger number of images. That is, as a result of the above,the probability that a captured image when the inclinations in the pandirection and the tilt direction are within a predetermined range isincreased so that a larger number of images are set as reproductiontarget images.

Furthermore, in the fourth and fifth embodiments, a case in which adetermination as to the inclination and the setting of a reproductiontarget image are performed in response to a reproduction startinstruction has been exemplified. In addition, these processings may beperformed automatically in a case in which, for example, the processingof the system controller 2 and the CPU 31 is comparatively idle, so thatthe setting of the reproduction target image can be completed before thereproduction start instruction. As a result of the above, it is possibleto shorten the time period from when a reproduction start instruction isissued until image display is started and also, it is possible toshorten the time period for the user to wait from when operation isinput until the display is started.

Furthermore, in the description thus far, a case in which theinformation processing apparatus constituting the image display systemaccording to the embodiment of the present invention is a personalcomputer has been exemplified. In addition, for the informationprocessing apparatus according to the embodiment of the presentinvention, for example, another information processing apparatus, suchas a PDA, may be used.

Furthermore, for the display control apparatus according the embodimentof the present invention, another apparatus other than theimage-capturing apparatus and the personal computer may be used.

It should be understood by those skilled in the art that variousmodifications, combinations, sub-combinations and alterations may occurdepending on design requirements and other factors insofar as they arewithin the scope of the appended claims or the equivalents thereof.

What is claimed is:
 1. An information processing apparatus comprising: circuitry configured to: estimate a traveling direction of a wearable camera; and obtain, based on an angle between the estimated traveling direction and a capturing direction of the wearable camera, first image data corresponding to a first image captured by the wearable camera in the capturing direction.
 2. The information processing apparatus according to claim 1, wherein the circuitry is configured to control the wearable camera to automatically capture the first image in the capturing direction based on the angle while the wearable camera moves in the estimated traveling direction.
 3. The information processing apparatus according to claim 2, wherein the first image is an image captured only when the angle is a threshold value or less.
 4. The information processing apparatus according to claim 1, wherein the circuitry is configured to obtain pieces of the first image data from second image data corresponding to a second image automatically captured by the wearable camera while the wearable camera moves in the estimated traveling direction.
 5. The information processing apparatus according to claim 4, wherein the second image includes an image captured when the angle is larger than a threshold value.
 6. The information processing apparatus according to claim 4, wherein the circuitry is configured to send pieces of the obtained first image data to a reproducing device, the pieces of the obtained first image data including no image data corresponding to an image captured when the angle is larger than a threshold value.
 7. The information processing apparatus according to claim 1, wherein the wearable camera is a life-log camera configured to automatically capture at least a scene in a field-of-view direction of a user wearing the life-log camera.
 8. The information processing apparatus according to claim 1, wherein the wearable camera constitutes at least a part of a neck-hanging apparatus, a head-mounted apparatus, an ear-mounted apparatus, or an apparatus having a clip.
 9. The information processing apparatus according to claim 1, wherein the angle is defined with respect to a pan direction or a tilt direction of the wearable camera.
 10. The information processing apparatus according to claim 1, wherein the circuitry is configured to estimate the angle based on information sent from at least one of a gyro sensor, a gravity sensor, and a direction sensor.
 11. The information processing apparatus according to claim 1, wherein the circuitry is configured to estimate the angle based on past positions of the wearable camera.
 12. A method of controlling a wearable camera, the method comprising controlling the wearable camera to automatically capture an image in a capturing direction of the wearable camera based on an angle between a traveling direction of the wearable camera and the capturing direction while the wearable camera moves in the traveling direction.
 13. A method of controlling a reproducing apparatus, the method comprising controlling the reproducing apparatus to reproduce pieces of first image data selected from second image data based on an angle between a traveling direction and a capturing direction of a wearable camera while the wearable camera is automatically capturing a second image, wherein the second image data corresponds to the second image, and a quantity of the second image data is larger than that of the pieces of the first image data.
 14. A method of recording image data, the method comprising recording, based on an angle of a traveling direction and a capturing direction of a wearable camera, image data corresponding to an image automatically captured by the wearable camera in the capturing direction while the wearable camera moves in the traveling direction. 